(1) Field of the Invention
The present invention relates to a reconfigurable neural network system.
(2) Description of the Related Art
Methods of performing cognitive signal processing such as face recognition and sound recognition utilizing a neural network are widely known.
The neural network simulates a signal processing system constituted of a neuron network in a human's brain, to perform neural signal processing in which mutually connected neuron cells operate in parallel. Such a process allows objects that are difficult to formularize, such as data containing a noise and a variety of faces and voices, to be flexibly and rapidly recognized. The neuron cell is a simulated model of a neuron structure, and serves to perform the neural signal processing in connection with other neuron cells or input devices. The neuron cell receives as input signals the output results of other neuron cells or input devices connected thereto, and performs an operation (reaction) in response to a specific input, in accordance with weight information of the respective input signals. Thus, a desired operation can be performed.
FIG. 13A schematically illustrates a popular mathematical model of the neuron cell. The mathematical model of the neuron cell shown in FIG. 13A operates in accordance with the following equations (1) to (3).
                    y        =                  f          ⁡                      (            u            )                                              (        1        )                                u        =                                            ∑                              i                =                1                            6                        ⁢                                                  ⁢                                          w                i                            ⁢                              x                i                                              -          T                                    (        2        )                                          f          ⁡                      (            u            )                          =                  1                      1            +                          exp              ⁡                              (                                  -                  u                                )                                                                        (        3        )            
In FIG. 13A, reference numerals x1 to x6 represent input signals from other neuron cells or input devices. Numerals w1 to w6 are quantities indicating combination efficiency of the input signals, and represent a weight with respect to x1 to x6. Value y is an output signal determined on the basis of the plurality of input signals x1 to x6.
In this mathematical model of the neuron cell, when the plurality of input signals x1 to x6 is inputted, the input x (x1 to x6 in this case) is multiplied by the weight w given to the respective input x (w1 to w6 in this case), and then u is obtained by subtracting an offset T from the total amount of the multiplication products, as shown in the equation (2). Here, the offset T corresponds to a threshold to determine whether the neuron cell is to react to a specific input.
Then a value obtained by substituting u calculated as above in the activation function f(u) expressed as the equation (1) constitutes the output y of the neuron cell. In this model, the sigmoid function expressed as the equation (3) determines the level of the output y. Specifically, y up to the threshold is outputted at a Low level, and y exceeding the threshold is outputted as a High level.
FIG. 13B is a graph showing a curve defined by the sigmoid function. The sigmoid function has asymptotic lines to y=0 and y=1 as shown in FIG. 13B, and the value of y remains within the range of 0 to 1 with respect to all values on the x axis. The sigmoid function also has an inflection point, and the curve is point-symmetric with respect to the inflection point. This also indicates that y up to the threshold is outputted at a Low level, and y exceeding the threshold is outputted as a High level.
Referring now to FIGS. 14A to 14D, typical structures of the neural network will be described.
FIG. 14A is a diagram showing a typical structure of a multilayer network.
The multilayer network refers to a network constituted of a plurality of neural layers, namely an input layer, an intermediate layer, and an output layer in which the neuron cells in the same neural layer are not connected, but the neuron cells in different neural layers are mutually connected.
The structure shown in FIG. 14A constituted of the three neural layers is known as perceptron, and is suitable for a pattern matching process.
FIG. 14B is a diagram showing a typical structure of a mutual connected network.
The mutual connected network refers to a network in which the neuron cells are mutually connected (coupled), instead of forming layers as in the multilayer network.
The structure shown in FIG. 14B is called Hopfield network in which the neuron cells are mutually connected, and has a characteristic that imperfect data containing a noise can be complemented and associated with a pattern in memory.
FIGS. 14C and 14D are diagrams showing other typical structures of the neural network.
The structure shown in FIG. 14C is called Elman network, in which feedback is made from another neural layer branched from the intermediate layer of the perceptron. The Elman network has a recursive structure that feeds back past information, and is hence suitable for recognizing a sequence pattern such as a grammar.
In addition, a central pattern generator (CPG) shown in FIG. 14D is also known, in which feedback is mutually performed between two neural layers. This structure is applied to posture control of a biped walking robot (see NPL 1: Jiang Shan, Fumio Nagashima, “Biologically Inspired Spinal Locomotion Controller for Humanoid Robot”, 19th Annual Conference of the Robotics Society of Japan, pp. 517-518, 2001).
As described above, characteristic operations of the respective neural networks are determined on the basis of the combination of the network configuration and the weight information.
Now, the neural network can be implemented in the form of either software or hardware.
Implementing the neural network by software is not suitable for real-time processing because the neural network model has to be emulated using an ordinary computer, and hence software is employed, for example, for searching a huge database (see NPL 2: Fukushima, Kurahara, Torikoshi, et al., “Development and Evaluation of Internal Diagnosis Support System Utilizing a Neural Network”, Lecture Article No. 431 in 18th Kumamoto Pref. Industry-Academia-Government Technology Meeting).
In contrast, implementation by hardware allows real-time processing to be performed, and is hence employed in image recognition systems. An implementation example of the neural network by hardware will be described hereunder.
FIG. 15 is a block diagram of a processor in which the neural network is implemented.
The processor shown in FIG. 15, called a ZISC processor, has a trilayer network configuration for performing cognitive processing (see NPL 3: IBM ZISC036 Neurons USER'S MANUAL Version 1.2). In this processor, the weight information can be set not only by a register but also by a self-learning mechanism, to obtain a desired operation.
The self-learning mechanism can be typically exemplified by weight information updating utilizing backward propagation. FIGS. 16A and 16B are diagrams showing an outline of a backward propagation process. As shown therein, in the backward propagation a teacher signal and reference data, which are ideal output signals, are given for error calculation in each neural layer, and the weight information is updated so as to minimize the error, to thereby accomplish a desired learning.
Further, an image recognition system utilizing the ZISC processor is disclosed (see PTL 1: Japanese Unexamined Patent Application Publication No. 2001-014470). FIG. 17 is a block diagram of the image recognition system that employs the ZISC processor. As shown in FIG. 17, the ZISC processor includes on its input side a data conversion processor that converts various image data into generalized data. Such conversion allows various data format to be processed using a single neural network configuration and the weight information.